Impact of hospital internships on success in university summative objective structured clinical examinations: Large-scale experience in a French medical school

Background Objective structured clinical examinations (OSCEs) are extensively used in many medical schools worldwide with the stated objective to assess students’ clinical skills acquired during internships. The objective of the present study was to assess the factors associated with success in university summative OSCEs, especially the impact of previous hospital internships in corresponding disciplines and supervision during internships. Methods This was a cross-sectional study assessing the results in the summative OSCEs of 4th year medical students during the 2021–2022 academic year in a French medical school. The summative OSCEs included five stations for each student. Each student answered a survey at the end of summative OSCEs about previous internships, the supervision they had and perceived difficulty levels for each station. The scores in each station were assessed according to previous hospital internships in the corresponding discipline. Analysis of predictive factors of success in OSCEs, defined by a score ≥ 10/20 at each station, were performed using univariate and multivariate logistic regression. Results Out of the 220 students participating in the summative OSCEs, 182 (83%) answered the survey. One hundred and forty-four (79%) of these students had carried out hospital internships in at least one of the disciplines evaluated during the OSCEs. Students having completed an internship in the corresponding discipline had significantly higher OSCEs scores for interrogation, communication, therapeutic education and procedure stations compared to those who had not. Previous internship in corresponding disciplines was independently associated with success in OSCEs in interrogation (OR 9.45 [1.34–66.8] p = 0.02), clinical examination (OR 6.93 [1.88–25.57] p = 0.004, and therapeutic education (OR 3.09 [1.22–7.82] p = 0.02) stations. Conclusion Previous hospital internships in the discipline evaluated by the OSCEs are associated with success in summative OSCEs. This reinforces the importance of student involvement during their hospital internships.


Introduction
To date, objective structured clinical examinations (OSCEs) are extensively used in many medical schools worldwide and are being progressively developed in France [1,2].Up until recently, assessments of medical students in France were based only on a written exam with multiple choice questions, which do not assess students' clinical skills or prepare them for clinical practice [3].To address this issue, a reform of the 2 nd cycle of medical studies in France in 2021 introduced OSCEs in the assessment of French medical students, with the first nationwide OSCEs planned for 2024 [4].The results of the national summative OSCEs will be part of the results used to classify students at the end of their 6th year and consequently enable them to choose their specialty.It will increase the discriminatory capacity of current evaluation modalities in French medical schools [5] insofar as it has been shown to be an efficient tool to assess the clinical performance of medical students [6].The stated objective of the reform is to assess students' clinical skills acquired during hospital internships in view of enhancing their investment in the latter and of better preparing them for their future clinical practice.Indeed, from the 4 th to the 6 th year of medical studies in France, medical students spend 50% of their time on placements in hospital departments, generally for periods of 3 months, where they provide clinical follow-up of patients, work in teams with senior doctors and gain enhanced clinical experience in the discipline concerned.Training and supervision during these internships are not standardized, and their impact on success in university OSCEs has not been studied in France or in the international literature.Identification of factors associated with success to the OSCEs is aimed at developing pedagogical avenues to improve student preparation.
The objective of this study was to assess the factors associated with success in summative university OSCEs, and more particularly the impact of previous hospital internships in corresponding disciplines.

Methods
This was a cross-sectional study aimed at assessing the relationship between the results of the summative OSCEs of 4 th year medical students (2021-2022 academic year) in the French medical school of Poitiers university and the results of a survey on OSCEs and previous hospital internships collected at the end of these OSCEs, and the scores obtained for the final faculty tests throughout the same academic year.

Organization of the summative OSCEs
The summative OSCEs took place on May, 31 th 2022 for 220 4 th year medical students and included a route of five stations for each, randomly assessing disciplines they studied during the academic year.The OSCE disciplines and subjects differed between the morning and the afternoon, evaluating similar skills, the objective being to avoid communication of the subjects between students, with balanced overall difficulty between the two half-days (Fig 1).
Simulated patients, trained beforehand so as to ensure evaluation homogeneity, were present at the interrogation, clinical examination, and therapeutic education stations.Each station lasted 7 minutes, during which the students read and followed the instructions.A scoring checklist validated by the educational committee was completed by two evaluators present in the room, with a final OSCE grade of 20.The two evaluators were asked to carry out their evaluation each separately.At the end of the OSCE station, they were asked to use the 2-minute transition time to reach a consensus by reviewing the differences between each evaluation grid.The consensus score was entered into the computer system.The score was normalized according to the number of items evaluated.OSCE grades were collected after each student's OSCE.

Survey on OSCEs and previous internships
At the end of the OSCEs, students were asked to answer a survey collecting the following information: places of hospital internships carried out during the 3 rd and 4 th years of medical studies and whether they had made formative or summative OSCEs for each internship, whether they had performed clinical examinations supervised by a senior, and whether they had participated in consultations in an out-patient clinic.Perceived difficulty level for each station using the Likert scale (0-5) was also collected.

Ethics
This study was carried-out in accordance with French regulations.The approval of an ethics committee or the consent of the participants was not required as this study was not involving the Human person according to the French Public Health Code (Loi Jarde ´-n˚2012-300 of March the 5th 2012, in application in November 2016-Article R1121-1: research conducted exclusively from the processing of personal data is outside the scope of the researches involving human subjects (RIPH)).The survey was validated by the data protection delegation of the University of Poitiers (n˚2022.75).All the data were self-reported by the participants with their written agreement on the questionnaire.Students were free to respond and individuals who did not participate had the same learning opportunities, and their exams assessed in the same way as those who did not participate.The assessment was independent of grading and pseudonymised with the student number.Confidential information was provided as defined by the European General Data Protection Regulation.

Statistical analysis
Categorical variables were expressed as frequencies and percentages and continuous variables as means and standard deviations.Categorical variables were compared using the chi-square test and continuous variables using the Student's t-test.Pearson correlation matrix was used to evaluate the relationship between variables.Success at a OSCEs was defined by a score �10/20, the minimum score required for validation of summative OSCEs.OSCEs results were analyzed according to the discipline, or the clinical competency assessed.Analysis of predictive factors of success in OSCEs was performed through univariate and multivariate logistic regression with a backward selection of included variables employing a threshold of p<0.20.Included variables were previous internship in the discipline (in 3 rd or 4 th medical year), activities in the internships (supervised clinical examination participation in consultations, OSCEs), perceived difficulty of the station, final faculty test results in the discipline, and gender.The significance threshold was set at p<0.05.All statistical analyses were performed using GraphPad Prism software (v.9, GraphPad Software, Boston, USA) and MedCalc (v.20.013;MedCalc Software, Ostend, Belgium).

Population characteristics and previous internships
Out of the 220 students, 182 (83%) answered the survey and were included in the study, of whom 112 (61.5%) were women.Thirty-eight (20.9%) had not carried out internships in the disciplines evaluated during their OSCEs, while 71 (39.0%) had carried out an internship in one of the disciplines evaluated, 52 (28.6%) in 2 different disciplines and 20 (11.0%) in 3 disciplines.Medical activities during hospital internships are reported in Table 1.Among the 182 students, whatever the discipline, 87 (47.8%) had previously performed �1 formative OSCE,

Performances during the OSCEs
The overall average OSCEs score was 11.3 ± 2.0 /20.Procedure and therapeutic education OSCEs were respectively the most (14.6/20 ± 3.2) and the least (8.6/20 ± 3.3) successful.The scores obtained in each station are detailed in Table 2.

Impact on OSCEs scores of prior internships in a given discipline
The scores obtained in the rheumatology (p = 0.0002), pneumology (p = 0.0294), geriatrics (p = 0.0175), endocrinology (p = 0.0004), psychiatry (p = 0.004) and cardiology (0.0011) were higher if the student had completed an internship in the discipline (Fig 2).
Similarly, when OSCEs station were grouped by evaluated competency, the difficulty levels perceived by students having completed an internship in the corresponding discipline were lower compared to those who had not done so, in the interrogation (p = 0.0004), communication (p<0.0001), and therapeutic education stations (p<0.

0001) (S2 Fig).
There was an inverse correlation between the difficulty level perceived by the students and the scores obtained in the interrogation (p = 0.0307), communication (p = 0.0025), therapeutic education (p<0.0001) and procedure (p = 0.0063) domains.Conversely, students having performed previous formative or summative OSCEs during internships, whatever the internship, did not have significantly different overall OSCEs scores compared to those who had not done so (11.1 ± 1.9 vs. 11.5 ± 2.1, p = 0.16 and 11.4 ± 2.1 vs. 11.1 ± 1.67, p = 0.47 respectively).The same results were observed for each clinical domain.

Predictive factors of OSCE success
Univariate analysis of predictive factors of success in OSCEs (score � 10/20) for each station and each discipline are reported in S1 and S2 Tables.In multivariate analysis, success in OSCEs was associated with previous internship in the corresponding discipline for the interrogation, clinical examination and therapeutic education stations (OR = 9.45 [1.34-66.80],p = 0.0244 OR = 6.93 [1.88-25.57],p = 0.0036 and OR = 3.09 [1.22-7.82],p = 0.0169 respectively) (Table 3).The predictive factors of success to OSCEs stations for each discipline are reported in Table 4.

Discussion
OSCEs have become an established clinical assessment tool in many countries worldwide and have been developed more recently in French medical studies.Our study is one of the first in Europe and worldwide to study the impact of hospital internships for success in summative university OSCEs.It shows that OSCEs grades are better and the level of perceived difficulties lower when students have carried out an internship in the corresponding discipline.Previous internships were an independent factor of success in OSCEs in interrogation, clinical examination, and therapeutic education stations.

Positive impact of prior internships on OSCEs scores
The relationship between better results in OSCEs and lower perceived difficulty in the station if previous internship was done in the discipline has previously been described for psychiatry with higher OSCEs scores in the event of a specialized internship [7].Indeed, the mental status examination, phenomenology, and differential diagnosis are probably more easily apprehended by the student and better mastered if they have been exposed to these diagnoses during internships rather than by learning during lessons or in textbooks [7].In our study, this observation was corroborated for all the clinical domains evaluated, except for OSCEs scores in clinical examination stations.The two disciplines evaluated in the latter were neurology and rheumatology, and the correlation was significant in the latter, but not in the former.Neurology is taught at an earlier stage of studies and is favored by students as an important [8] but difficult discipline [9].In France, medical students carry out several full-time 3-month hospital internships in a given discipline, during which they receive clinical training and carry out, independently and under direct supervision, clinical examinations every day.Daily training with actual patients concretely prepares students for clinical practice, and their learning and practice of fundamental skills can help to explain its being a major success factor in completed specialized internships.This finding is supported by a study which shows that OSCEs efficiently assess the clinical performance of medical students [6].

Lack of impact of formative and summative OSCEs sessions during internships on OSCEs score
Surprisingly, OSCEs success was not associated with completion of the formative and summative OSCEs sessions organized during the various internships, even though OSCEs have been systematized in most hospital internships.This may be explained by differences and lack of homogeneity in the conduct of OSCEs during internships, which are quite new in France, which means that the relevant training of teachers as well as students has remained heterogeneous, and consequently does not yet conclusively determine success in summative university OSCEs.

Positive correlation between OSCEs scores and faculty test results
In a study assessing the predictive factors for OSCEs success among 5 th -year medical students, female gender, absence of health problems during the internship, high number of internships in a medical specialty (�6) and low number of internships in a surgical specialty (<3) were the most significantly factors associated with higher OSCEs success.In this study, success in theoretical faculty exams had no effect [10].By contrast, in our cohort one of the factors of success in OSCEs was success in final faculty exams.On this subject, a correlation between faculty test results and OSCEs performance was clearly demonstrated in the musculoskeletal domain in a study by the Mexican Board of Rheumatology on postgraduate certification procedures [11].However, this correlation is not perfect, especially insofar as theoretical faculty exams and OSCEs do not assess the same skills.Another recent study showed a weak correlation between OSCEs grades and faculty exams among 6 th year medical students, but these latter obtained lower scores in OSCEs exams compared to other standard evaluation modalities, leading the authors to suggest that the OSCEs are not redundant with the other evaluation modalities [12].

Better results in procedural skills
The scores obtained in the clinical examination and therapeutic education stations were lower than those obtained in the procedure stations.Therapeutic education requires communication skills, which have been shown to be correlated with OSCEs performance evaluating clinical examination, problem solution prescription, and medical records [13].On one hand, performance of some procedures (the electrocardiogram. ..) are part of the daily lives of medical students.On the other hand, communication with the patient, therapeutic education, and performance of specialized clinical examinations, one example being examination of the knee in rheumatology, are skills acquired by only a minority of students.This is in line with a study showing that among the different skills evaluated by OSCEs, medical students are less effective at clinical reasoning skills, while better results are found in procedural skills [14].It is therefore appropriate to increase this form of skill acquisition in view of upgrading OSCEs results.The particularity of the musculoskeletal examination was previously described in the literature.In this domain, the addition of an interactive workshop with a focus on clinical examination of the knee in the rheumatology rotation of medicine residents led to significantly improved OSCEs performances [15].Homogenization of internships during medical training courses and relevant activities in the different disciplines help to improve skills acquired by medical students and are likely to improve grades obtained during the OSCEs.Having completed a specialized internship improves success, and the benefit of this continuous training has been underlined in the literature and should be encouraged [16][17][18].

Limitations
Our study has several limitations.First, it was conducted at a single institution.However, there was a high response rate (83.1%), which strengthened the results.Moreover, a survey is associated with potential information biases and a certain number of missing data, and result should be considered with caution.In these OSCEs, we used a checklist to determine the student's grade, but there were no global evaluation scales, which would have improved assessment according to previous internships, as previously shown [19].In addition, for each station, the two evaluators made a consensus for the final score, and we had no individual scores to assess reliability and generalizability.

Conclusion
One of the main factors associated with OSCEs success was participation to previous hospital internships in the discipline evaluated by the OSCEs.Homogenization and increased supervised medical activities carried out during the internship will render them even more educational.

Table 1 . Number of student participants in internships in disciplines evaluated during the university OSCEs. Internships, n (% of all sutdents) Formative OSCEs, n (% of students in the internship) Summative OSCEs n (% of students in the internship) Supervised clinical exams n (% of students in the internship) Consultations n (% of students in the internship)
�1 summative OSCE, 112 (61.5%) supervised clinical examinations, while 144 (84.6%) had participated in consultations during their internships.

Table 3 . Predictive factors of success in OSCE stations (multivariate analysis). Clinical domain of OSCEs
OSCEs: objective structured clinical examinations, CI: Confidence interval, ND: Not included in the multivariable model Logistic regression with backward selection of included variables using a threshold of p <0.20 https://doi.org/10.1371/journal.pone.0302427.t003